Models as Approximations — Part II: A General Theory of Model-Robust Regression

We discuss a model-robust theory for general types of regression in the simplest case of iid observations. The theory replaces the parameters of parametric models with statistical functionals, to be called “regression functionals” and defined on large non-parametric classes of joint x-y distributions without assuming a working model. Examples of regression functionals are the slopes of OLS linear equations at largely arbitrary x-y distributions (see Part I). More generally, regression functionals can be defined by minimizing objective functions or solving estimating equations at joint x-y distributions. The role of parametric models is reduced to heuristics for generating objective functions and estimating equations without assuming them as correct. In this framework it is possible to achieve the following: (1) explicate the meaning of mis/well-specification for regression functionals, (2) decompose sampling variability into two components, one due to the conditional response distributions and another due to the regressor distribution interacting (conspiring) with misspecification, (3) exhibit plug-in (and hence sandwich) estimators of standard error as limiting cases of x-y bootstrap estimators. AMS 2000 subject classifications: Primary 62J05, 62J20, 62F40; secondary 62F35, 62A10.